Coping with Sparsity in a Recommender System

In thispaper we report experimentsthatweconductedusing an implementation of ar ecommender system called " KnowledgeP ump " (KP) developed at Xerox. We repeat well-known methodss ucha st he Pearson method, but alsoa ddressc ommonp roblems of recommender systems, in particular thes parsityp roblem. Thes parsityp roblem is the problem of having toof ew ratings andh ence toof ew correlationsb e-tween users. We address this problem in twodifferentmanners. First, we introduce " transitivec orrelations " , am echanism to increase then umber of correlationsbetween existingusers. Second, we add " agents " , artificial users that rate in accordance with some predefinedpreferences. We show that both ideas payo ff, albeiti nd ifferentw ays: Transitivec orrelations provideasmall help for virtually no price, whereas ratingagents improve thec overage of thes ystems ignificantlyb ut also have an egativei mpact on thes ystem performance. Recommenders ystems have enjoyedg reat popularityd uring recenty ears and have been recognized as an importantt oolf or information filtering. They help userst og et personalized recommendations in as omed omain. Typicald omains include books, movies, andm usic. The strength of recommender systems comes from the fact thatt hey don't need to analyze the contento ft he items they recommend. Foru sersi ti sa lso possible to discovern ew items purely on the basis of positiver atingso fo ther users. Hence, recommender systems have been classified as socialfiltering (asopposed to cognitiveand economic filtering [10]). In thes implest case, recommender systems, also referred to as collaborative filtering systems, consist of ad atabasew ith three tables: users, items, andr at-ings. This database is used to find usersw hose opinions are similara nd make use of thatk nowledget oc ompute rating predictions fori tems thats omeu sers have rated ando thers have not. Recommender systems do suffer from some problems. The most obvious one is the sparsityp roblem.Ifthere are only afew ratingsfor every item then it is impossible to compute thec orrelation (the degree of similarity) between users. For example, at the time of writing the well-known andp opularm ovie " StarW ars " O.R. Zaï ane et al.